Evidence-Based Practices to Support 
College and Career Readiness in High School 


Early Warning Indicators 


The Every Student Succeeds Act (ESSA; 2015) requires states to identify high schools with a graduation rate 
less than 67% as low performing. Each low-performing high school is required to implement an evidence-based 
intervention to improve student outcomes. Although the number of interventions that are associated with improved 
outcomes for high school students is limited, early warning indicators represent an intervention with evidence. 
Figure 1 presents the four levels of evidence used by the U.S. Department of Education. Early warning indicators 
provide a means of screening and identifying 
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What Are the High School Indicators? 


The majority of early warning indicators are measures of attendance and course performance (Bruce, Bridgeland, 
Hornig Fox, & Balfanz, 2011; Li et al., 2016). These indicators have been identified through research to be 
highly predictive of student outcomes (Heppen & Therriault, 2008), such as on-time high school graduation. 
To successfully use these indicator thresholds, which are specific values that predict whether a student will 
meet the predetermined outcome of interest without intervention, need to be determined (Li et al., 2016). 
However, some variation exists in the accuracy or predictive power of these indicators across different grade 
levels and districts (Li et al., 2016). Frequently used ninth-grade indicators and their accompanying typical 


thresholds are detailed Table 1. 


Table 1. Frequently Used Ninth-Grade Indicators and Thresholds 
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The Evidence for Indicators 


Starting in the early 2000s, researchers from the University of Chicago Consortium on Chicago School Research, 
the Center for Social Organization at Johns Hopkins University, and the Philadelphia Education Fund began to 
examine the factors that were most predictive of students dropping out of high school prior to graduation 
(Heppen & Therriault, 2008). Since then, a strong research base has been established for the attendance 
and course performance indicators. Because behavioral data vary widely by location, such data have a 
weaker evidence base than the other indicators (Allensworth & Easton, 2007). 


Attendance. A strong correlation exists between attendance and on-time graduation (O’Cummings, 2015). 
Allensworth and Easton (2007) found that students with extremely high absenteeism rates (missing one 
month or more of class each semester) have less than a 10% chance of graduating on-time (O’Cummings, 
2015). However, moderate absenteeism rates can still be problematic; only 67% of students who missed 
1 week of school per semester proceed to graduate on-time (O’Cummings, 2015). However, attendance is less 
predictive than the on-track, GPA, and course failure indicators because the attendance indicator does not 
differentiate between students who are attending school consistently but performing poorly from students 
who are attending school consistently and performing well (O’Cummings, 2015). 


Course Performance. Course performance can be measured in a variety of ways, including course failures, 
credit accumulation, and grades (Allensworth & Easton, 2007). Ninth-grade performance was found to have a 
strong correlation with high school graduation, with almost all students (95%) averaging a B or higher at the 
end of their freshman year proceeding to graduate on time (O’Cummings, 2015). 


Composite Indicators. Allensworth and Easton (2005) found that the on-track indicator is highly predictive 
of on-time graduation, with students on track being nearly four times more likely to graduate on time than 
students who are off-track. However, composite indicators are not precise enough to provide actionable 

information to determine interventions for students (O’Cummings, 2015). 


State Policy Considerations 


States can support the use of early warning indicators through various strategies. States have incentivized the 
use of early warning indicators by validating such indicators for use by schools and districts, monitoring early 
warning indicators to identify needs and allocate resources, and collecting early warning indicator data from 
districts and schools. 


Validating Indicators. The thresholds presented in Table 1 are based on research conducted in Chicago, 
Illinois; Baltimore, Maryland; and Philadelphia, Pennsylvania. However, further research has found that some 
variation in the accuracy of cut points exists across contexts (Allensworth & Easton, 2007). By validating indicators, 
states can ensure that the data are more applicable and meaningful to their local contexts (O’Cummings, 
2015). In addition, validating indicators can increase the legitimacy, or predictive power, of the indicators for 
schools within a state (Allensworth & Easton, 2007). For example, the Minnesota Department of Education, 
the Wisconsin Department of Education, and the Massachusetts Department of Elementary and Secondary 
Education have each validated early warning indicators using student data in their own states (O’Cummings & 
Therriault, 2015). For more information on validating indicators, see the District Guide for Creating Indicators 
for Early Warning Systems (Li et al., 2016). 
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Monitoring Indicators. Using early warning indicators to monitor student progress require a larger implementation 
framework for providing supports to students. By examining early warning indicators at the student, school, and 
district levels, stakeholders can monitor student progress and identify needs. It is critical to monitor these indicators 
at the school level to identify students who are at risk and provide support, but it is valuable to look at these 
data at the district and state levels to identify trends in need and promising interventions and allocate resources 
to support larger scale improvement efforts (American Institutes for Research & Massachusetts Department of 
Elementary and Secondary Education, 2014; Therriault, Heppen, O’Cummings, Fryer, & Johnson, 2010). 


Formative Progress Measures. States and districts can encourage the use of early warning systems as part of 
a process of formative feedback on progress toward improving student graduation outcomes and as proximal 
or early measures for progress toward outcomes. For example, a state or district may require schools to provide 
information on the percentages of students who are at risk based on early warning indicators as part of a 
monitoring and reporting process (Blumenthal, 2016a). Early warning indicator data are predictive, yet some 
states are including the data as part of a state accountability process. For instance, the California Department 
of Education reviews high school graduation data as part of middle school reporting. As another example, one 
state education agency required grantees (districts) of a summer bridge program to report on early warning 
indicators, including attendance, course performance, GPA, and credits earned for all students who participated 
in the program through ninth grade (O’Cummings & Therriault, 2015). Another state education agency required 
each low-performing high school to report on the percentages of students at risk by each early warning indicator. 
Such programmatic mandates can be used as an entry point for a state to begin to engage districts and 
schools in using early warning indicator data. 
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